Skip to content

Size replies to the question, gate claims on results, and say the push to act once - #1538

Draft
davidmckayv wants to merge 8 commits into
mainfrom
prompt-pass/working-rules
Draft

davidmckayv wants to merge 8 commits into
mainfrom
prompt-pass/working-rules

Conversation

@davidmckayv

Copy link
Copy Markdown
Contributor

Every Bot's working rules, reworked against what dogfooding showed: replies that ran to 600 words for a yes-or-no question, "are you able to...?" answered by doing unasked work, "done" said before anything was checked, a Python script approved on a computer with no Python, and spreadsheets left plain.

What changes for every Bot

  • One push to act, bounded by what was asked. The push to act was said three times (<how_to_work>, the ask test's "keep going" and the Personal Assistant's role) with nothing about scope. It is now said once, in <how_to_work>: do the preparation a task needs without asking, take a decided step in the same reply, and do not start work nobody asked for, so a question gets its answer, not the job it is about. The ask test's else-branch now reads "carry on with what was asked".
  • A search that has not come back. A Bot that started a tool search waits for it before deciding, or saying, that something cannot be done.
  • Failures by kind. A temporary failure gets one retry; a declined step stays declined; a change whose outcome is unknown is looked for in the app before it is redone; a failure that keeps happening becomes a short report.
  • Claims need a tool result. "Done, sent, saved or correct" only when a tool result in this conversation showed it (Anthropic's own rule for agents handling records). After a change in an app, read it back once. Report scoped to what was checked; "all" only if every item was checked.
  • Worded per model. Anthropic's Opus 5 guide says that model reads its own work back unprompted and over-verifies when told to, so readsBack leaves the read-back sentence out for Claude Opus 5 and later; the claims rule stays for every model. GPT-6.x and Claude Sonnet/Haiku keep a bounded "once", which is OpenAI's own fix for GPT-6 testing too broadly. Gemini 3.x has no such guidance and keeps it.
  • <replies>, new: size the reply to the question and lead with the answer. Yes/no gets yes or no and at most one line more. A question about whether the Bot can do something gets a short answer from what it holds and an offer, not the work, not a read of the person's data, not a capability list. A request worded as a question ("can you send me today's agenda?") is a request. A finished job is one or two sentences with the result or a link. No recaps, no repeated summaries, no message per step. It says it is about words: a component the answer calls for is still drawn, so generative UI is unchanged.
  • <documents>, new: spreadsheets get labelled input cells, live formulas, units in headers, a bold frozen header row, real number, percent, currency and date formats, one tab for a small task, kept formatting, and a read-back for #REF!, #NAME? and #VALUE! before delivery. Documents get a title, headings and no filler.
  • What the computer has. computerGuidance takes an optional installed list from the deployment that builds the computer's image, and tells the Bot to check before writing a script and never to ask the person to approve a script for a program the computer does not have.
  • The "do it now" nudge lets an offer stand. Its words now hold the Bot to work it was asked for, and leave an offer for the person.
  • The Personal Assistant's role drops the repeated push to act and the duplicate ask test; it keeps what is particular to an assistant and adds "when they ask whether you can, answer and offer" and "keep replies short".

Tests

  • shared/bot-prompt.test.ts: the push to act appears once across every Bot's working rules and "keep going" nowhere; scope, search wait and typed retries; the claims rule; the read-back present for GPT-5.6, GPT-6.1, Sonnet 5.5, Haiku 5.5, Opus 4.8, Gemini 3.8 and unknown models, absent for Opus 5 and later; reply shapes including the request-worded-as-a-question case and the line that keeps components; spreadsheet defaults; the computer's software line only when given; section order; no shouting.
  • agent-claude-sdk/tests/fixtures/server-instructions.json regenerated, and test_openbot_instructions.py passes against it.
  • The server tests that pin the section order and the nudge's words are updated to the new ones.

Evidence from real-model runs, before and after, is below.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant